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Abstract 

In this paper we study multiagent models with time-varying type change. 
Assume that there exist a closed system of N agents classified into r types 
according to their states of an internal system; each agent changes its type by an 
internal dynamics of the internal states or by the relative frequency of different 
internal states among the others, e.g., multinomial sampling. We investigate 
the asymptotic behavior of the empirical distributions of the agents' types as 
N goes to infinity, by the weak convergence criteria for time-inhomogeneous 
Markov processes and the theory of Volterra integral equations of the second 
kind. We also prove convergence theorems of these models evolving in random 
environment. 
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1. INTRODUCTION 

Agent-based models (ABMs), or multiagent systems (MAS), arise from many areas 
of science and social sciences such as ecology, artificial intelligence, communication 
networks, sociology, economics; see e.g. Ferber (1999, 1995), Ouelhadj (1996), and 
Wooldridge (1995). Goldstone and Janssen (2006) studied the collective behavior of 
the agent-based computational models, which build social structures from the 'bottom- 
up'. We give some of the attractive features of ABMs presented in their paper. First, 
ABMs can describe precise mathematical formulation, which make clear, quantitative 
and objective predictions possible. Second, ABMs can bridge the explanations that 
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link the analysis of the individual agent level and the analysis of the emergent group 
level. 

In this paper we will focus on the agent based modeling in social or economic 
discipline. Follmer and Schweizer (1993) considered an interacting agent financial 
model in which they used Black's (1986) classification of traders: information traders 
and noise traders. Lux (1995, 1997, 1998) studied a model of three types of traders 
which can probabilistically change their types. Horst (2000, 2001, 2002, 2005) kept 
some aspects of Follmer and Schweizer's model and considered interacting agent models 
with local and global interactions. Horst assumed that the set of active agents A is 
countable and there is a sequence of finite sets A n satisfying lim^oo A n = A. In 
Horst's example, the traders are divided into fundamental traders and noise traders, 
and the fundamental traders are divided into optimistic and pessimistic fundamental 
traders. Horst introduced the concept of individual mood into his models. At each 
period t > 0, each fundamental trader has its own mood, e.g., x% = +1 or xf = — 1, 
that is to say, the fundamental trader is an optimist or a pessimist. Let C be a fixed 
set of individual states, i.e., if e C for each a e A and t > 0. Let x t = {x^} ae ^. 
Horst defined the empirical distribution, which is called mood of the market, as follows: 

Pt = p{x t )= lim M')- 

The market mood is one of the main driving forces of Horst's interacting agent models. 

So far, we have seen the importance of the empirical distribution of the individual 
states which links the behavior of individual agent level and the emergent laws of 
collective level. The multiagent models of this paper arise from social or economic 
background; some features of them show similarities with the Wright-Fisher model in 
population biology , and the Voter model, see e.g. Either and Kurtz (1986), and Holley 
(1975). Instead of giving the precise definition of the agents, we describe the properties 
and behaviors of the agents rather intuitively. The multiagent models here share some 
similarities with ABMs in other disciplines. The most general assumptions about the 
mechanism of the multiagent models are as follows: 

1. The time is in nonnegative integer units, denoted by k > 0. 

2. There are fixed N >2 agents in the multiagent system at all times. There are no 
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entries of new agents into the system or exits of current agents from the system. 

3. There is an internal system with which all agents are concerned. The internal 
system has r > 2 states which we simply denote by 1, • • • , r. The internal system 
will not change with time k. That is to say, at any time k > 0, there is no new 
state added to the internal system and there is no existing state removed from it. 
The behavior of internal states are observed by all agents. Each agent has one 
and only one internal state at each time k > 0. Thus, the agents are classified 
into r types, according to their internal states. 

4. Assume that nf(k) (1 < i < r) is the number of agents of type i at time k and 
n N (k) — (n^(k), (k), ■ ■ ■ , (k)) is the distribution of all agents among the 
r types. By the second assumption, n^(fc) + • • • + n 1 ^ (k) = N and 11 ^ is the 
empirical distribution of the types at k > 0. 

5. Assume that {{pN,ij(k)) r xr, k > 0} is a sequence of deterministic stochastic- 
matrix valued functions which represent the external environment of the multia- 
gent system. 

6. Based on all the information of the agents' types and the environment up to time 
k, each agent has an independent strategy of probabilistically choosing its type 
for the next time unit k + 1. The strategy of an agent is realized by keeping or 
changing its type. The agents of the same type have a common strategy. That is 
to say, from time k to fc + 1, the agents of type i switch to type j with probability 
PN,ij(k)- This process of changing types occurs locally among agents of the same 
type, and it is called internal dynamics. 

7. From time k to k + 1, there also occurs another process of global type change. 
When we make this assumption, we would make a minor change on the fifth 
assumption, to say that the internal dynamics occurs from time k to k + |, 
rather than from time k to k + 1. Based on all the information up to time k + |, 
each agent independently determines its type by - ^ + 2 - . That is to say, for any 
agent, regardless of its type at time k + \ , the probability of its new type being 



i at time k + 1 is 



l£(*±i) 



N 



. Therefore, n N (k + 1) ~ multinomial! N, 
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Note that we can change the number i in the seventh assumption by any number c 
(0 < c < 1). The external environment in the fifth assumption can be external economic 
fundamentals. An example of this is given by Example ^ Based on the assumptions 
1-6, we can construct the multiagent model with internal dynamics (MAMWID). If 
we have all the seven assumptions, we can construct multiagent model with internal 
dynamics and multinomial sampling (MAMWID AMS). The multinomial sampling is 
a kind of interaction among the agents. When we assume that {(pN,i,j(k)) rxr , k > 0} 
is a random sequence, we can construct multiagent model with internal dynamics and 
random environment (MAMWID ARE); and multiagent model with internal dynamics, 
multinomial sampling, and random environment (MAMWID AMSARE) respectively. 

In this paper, we will mainly study the asymptotic behaviors of the empirical 
distribution, {— — ^ vt ^ , t > 0}, of the types, as N — > oo. Lux assumed the number 
of the agents to be finite, but he didn't consider the asymptotics of the empirical 
distribution of the types as N becomes large. Follmer, Schweizer, and Horst assumed 
that the number of the agents is countable; they do not have the question we discuss 
here. Another feature of our multiagent models is that the transition structure of the 
internal dynamics is time-inhomogeneous. 

This paper is the first attempt of a systematic study of the interacting agent financial 
systems. Another working paper of the author, which goes one step further than this 
one, focuses on the interacting agent feedback finance models, see Wu (2006). 

This paper is organized as follows. In Subsection 12. II we formulate MAMWID and 
MAMWIDAMS; and state their convergence in Theorem 12.11 In Subsection 12.21 we 
formulate MAMWID ARE and MAMWIDAMSARE; and state their joint and annealed 
convergence in Theorem 12.21 In Section [3J we prove Theorem 12.11 and in Section 4, 
we prove Theorem 12.21 In Appendix El we state weak convergence criteria for time 
inhomogeneous Markov processes, which are the main tools of this paper. 

2. Formulation of the Multiagent Models and Main Results 

2.1. Multiagent models in deterministic time-varying environment 

Now we formulate the multiagent model with internal dynamics (MAMWID) based 
on the assumptions 1-6 in Section^ Let A(t) = (di,j(t)) r xr be a r x r matrix-valued 
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cadlag function on [0, oc), which satisfies the conditions 

1) For each t > 0, A(t)e = rx i, where e = [1, • • • , 1]' and '"" denotes transpose; 

2) For each t > 0, 1 < i,j < r, i ^ j, a h3 (t) > 0. 

Fix N > 1, let Afj{i) = {aN,i,j(t)) rX r be a r x r matrix-valued function on [0, oo), 
which satisfies the conditions 

1) For each t > 0, AAr(i)e = rx i; 

2) For each t > 0, 1 < i,j < r, i j, a N ,i,j(t) > 0; 

3) An is a stepwise function, i.e., Apf(i) is a constant on M^, ^i^) for each fe > 0. 

Therefore and {Ajv(i)} are Q-matrix valued functions. 

Let R rxr be the Euclidean space corresponding to the rxr square matrix. For each 
N > 1, and fc > 0, let 

1 

PjV.fc = (j>N,i,j(k))rxr =-?+ — ^4at( — ), (2.1) 

where I is the identity matrix of order r, and PN,i,j(k) is the probability of each agent 
of type i switching to type j at time k + 1. The definition of P/\r,fc is valid since for 
large enough N, Pjsr,k is a stochastic matrix, which we call internal transition matrix 
of MAMWID. 

We are ready to formulate MAMWID. For k > 0, the transition between n N (k) and 
rr^fc+l) is determined by the sixth assumption in Section^as follows. For 1 < i < r, 
each agent of type i can change its type to j, with probability PN,i,j(k) (1 < j < r). 
Since the nf (k) agents independently make their transitions, the distribution of these 
nf (k) agents among the r types at time k + 1 is a random vector denoted by &N,k,i — 
(&v,fc,i,i ; ' • ■ ,^N,k,i,r), which satisfies 

Sjv,t,i ~ multinomial(nf (k), P N ,k,i-), ( 2 -2) 

where Pn kv is the i-th row of the matrix Pjv &■ Since agents in different type change 
their types independently, 3jv,fc,i, • • • , Ejv;fc,r are independent. The distribution of all 
the agents at time k + 1 is 

n N (k + 1) = Hjv.fe,! + • • • + S N . kir . (2.3) 
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The sequence {n N (k), fc > 0} defined this way is a time inhomogeneous Markov chain. 
At last, we define 

n N ([Nt}) 



X A (i) 



N 



(2.4) 



We introduce some notations. We put Z+ = {0, 1, ■ • • } and = [0, go), 

r 

K N = {N^a : a G (Z+) r , ^ a t = N}, 



i=l 



and 



K={a:ae(R+) r , ^^ = 1}. 



We define the time-dependent generator {GA(t),0 < t < 00} on C 1 (if): for each 
/ G C l (K) and t > 0, 

G A (t)f( P ) = pA(t)?f'. (2.5) 
ax 

It is clear that, for each t > 0, 9{G A {t)) = ^{K) and 9{G A ) = C Y (K) is the 
common domain of the generator {G A (t), < t < 00}, and D — C 2 (K) is a subalgebra 
contained in @>(G A ). 

Next, we illustrate the transition structure of the multiagent model with internal 
dynamics and multinomial sampling (MAMWIDAMS) by the following diagram: 











n«(fc+l) 


N 


Internal Dynamics 
Pnm 


N 


Multinomial 
Sampling 


N 









Figure 1: The transition structure of MAMWIDAMS 

Once {n N (fc), fc > 0} is defined by the Figure Q we define 

Y N (f) n N ([Nt]) 
W- N ■ 

Next, we define the differential operator Gb on C 2 (K) by 

d 2 



Gb = \Y. Mp) 



dxidxj 



(2.6) 



(2.7) 



where &y(p) = Pi(§ij — Pj)- We also define the generator {G A B{t),0 < t < 00} on 
C 2 (if) by 

G A ij(t) - G A (t) + G B , for any t > 0, (2.8) 
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whose common domain is @(G A b) = ^{Ga) H 2>{G b ) = C 2 (K). 
Define the metric djj on D RrX r[0, oo) as follows: 

/■oo 

du(x,y)= e~ u sup [||x(t) - y(f)|| r A l]du, x, y6%x,[0,oo). (2.9) 

JO 0<t<« 

Theorem 2.1. Define P N , k by fO)) . Le^ G ^(iT). ,4 ssume thatYanN ^oo djj {An , A) = 
0, where du is defined by \2.y\) . 

1) MAM WW. Define {n N (k),k > 0} &?/ i/ie internal dynamics. Define X N 
by \2.4[ l- If -P(X Ar (0)) _1 =>■ /i, then there exists a unique solution X°° of the 
Dk [0, oo) martingale problem for (Ga, p) on C 2 (K), and ~X. N X°°. 

2j MAMWIDMS. Define {n N (k),k > 0} &y £/ie Figure^ Y N by and 
G J 4s(i) by fSH.ty) . If P(Y JV (0)) _1 => /.t, £/ien i/iere exists a unique solution Y°° 
o/ the Dk [0, oo) martingale problem for (Gab, p) 071 C 3 (-^Q; and Y w =>■ Y°°. 

2.2. Multiagent models in random environment 

In this subsection, we assume that {^4jv} and A are random elements which rep- 
resent an external random environment. We also assume that A is Cfjrxr [0, oo)- 
valued. Then, we need the condition limjv^oo d(A^, A) — instead of the condi- 
tion lim^v-foo du(Apf, A) — 0, where d is the complete separable Skorohod metric on 

Let (Jzf, aV) be the subspace of (D Rr x r [0, oo), du) such that each element of Jz? 
satisfies the conditions at the beginning of subsection l2.ll Let (J£ c , djj) be the subspace 
of (C R rxr[0,oo),du) such that _Sf c = «5f f| C R r*r [0, oo). Let (^(£> K [0, oo)), p) be 
the space of probability measures on £)^[0, oo) where p is the Prohorov metric on 
S?(F>k [0, oo)). Then Theorem 12.11 shows that for any A G Jt? c , there exist unique 
Pa G 2?(Dk^, oo)) such that under Pa, the coordinate process Z on Dk[0, oo) is the 
unique solution of the martingale problem for the generator {GA(t),0 < t < oo} on 
C 2 (K), and Pab G ^(-Dif[0, oo)) such that under Pab, Z is the unique solution of 
the martingale problem for the generator {GAB{t),0 <t< oo} on C 3 (K). We define 
$ : J^ c h-» ^(£) K [0,oo)) by = P A and * : «5f c ^ ^(L> K [0,oo)) by = 

For each AT > 1, let (J2?at, d[/) be the subspace of (Jzf , aV) such that for each G 
•Sfjv, A^(t) is a constant on t G [■jc'j^rO f° r each k > 0. Then for given v4at G 
Jdjv, there are unique probability measure Pa n G &(Dk n [0, oo)) which is related to 
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MAMWID, and unique probability measure Pa n b G ^{Dk n [0, oo)) which is related 
to MAMWID AMS. Under Pa n or Pa n b, the coordinate process Zn on Dk n [0, oo) has 
the same distributions as those defined for X N of MAMWID or Y N of MAMWID AMS. 
Then we can define § N : ££ N i-> &>(D Kn [Q,oo)) and V N : ££ N i-> ^ > (D Kn [0, oo)) 
correspondingly. 

Example 1. Assume that (H,dn) is a polish space. {h N (jj),k > 0} is a sequence 
of external economic fundamentals taking values in (H,dn)- We assume also that F 
is a continuous mapping from (Dh[0, oo), ds H ) onto (Jzf, dy), where dg^ represents 
the complete separable Skorohod metric on Dh[0, oo). Let yljv(t) = i ;l (/i JV (i^)), for 
any N > 1 and t > 0. Then {h N (-^-),k > 0} constitutes the environment of the 
multiagent system. Thus we can define our multiagent system which are driven by 
internal dynamics, interaction among agents, and external economic fundamentals. 
Now we have multiagent models evolving in random environment if we assume that 
{h N (jf ) 7 k > 0} is a random sequence. The idea of external economic fundamentals in 
deterministic or random environment was used by Horst (2000). 

We assume that for each N > 1, An is an Jzfjv-valued process defined on some 
probability space (£In, ^n, Qn), and A is an J<? c -valued process defined on (f2, J?", Q). 
Then we can define multiagent model with internal dynamics and random environ- 
ment (MAMWIDARE), and multiagent model with internal dynamics, multinomial 
sampling, and random environment (MAMWIDMSARE). 

Now, we state the joint and annealed weak convergence theorem for the multiagent 
models evolving in random environment. 

Theorem 2.2. (Joint and annealed convergence) Let {An} and A be defined 
above, [In £ ^{Kn) for each N > 1 and ^ E &(K). Assume that An => A, and 

1) MAMWIDARE. ForN > 1 andui N E Q;v, define <f> N (A N (u N )) G &{D Kn [0, oo)) 
such that $Ar(A7v(wAr))(^Ar(0)) _1 = (In! for each w G O, define <b{A{ui)) E 
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&>{D K [Q,oo)) such that ®(A{u)))(Z(Q))- 1 = //. Then 

(A N ,$ N (A N ))^(A,$(A)) (2.10) 
Urn p( / t>(A N (Lu N ))Q N (clio N ), f $(A(w))Q(dw)) = 0. (2.11) 

N^oo J J 

2) MAMWIDMSARE. ForN > 1 anduj N G fl N , defined N (A N (ui N )) e &>{D Kn [0, oo)) 
sitcft iftai x I'Ar(AAr(ijJjv))(2'jv(0)) _1 = Hn> for each uj S fi, define ^f(A(u>)) € 
^(DictOjOo)) sttc/i i/iat *(A(w))(Z(0))" 1 = /i. TTien 



(2.12) 



lim p( / *(A JV ( Wjv ))Q JV ( ( iu; J v), / *(A(w))Q(dw)) = 0. (2.13) 



3. Proof of Theorem Wl\ 

At first, we consider some properties of {n N (k),k > 0} defined just by internal 
dynamics. Let V = [v%, ■ ■ ■ ,v r )' be a positive vector. For any k > 1 and m > 0, by 
and \2.ty . the independence ofSjf,k,i % an d the Markov property o/{n (fc), k > 
0}, we have 

fc-1 x nf (to) 



n 



nf (m+fe) 



n (m) 



i=l ^ 1=1 ' 



(3-1) 



where Jli=i PN,m+i = PN.m+i x ••• x P/v,m+fc-i- W^e make the convention that 
when we denote the product of a sequence of matrices by prod, we actually make the 
multiplication from the left to the right as the index increases its order. 
Then it follows by 1^3.1)) that 



m+fe— 1 

E[n N (m + k)\n N (m)]=n N (m) JJ p N,u 

l—m 



and for 1 < j < r, 



(3.2) 



E[nf (m + k)(nf (m + k) - l)|n iv (m)] 

m+fc-2 -| \ 2 

n ^ 



n N (m) 



N,m+k-l,-,j 





rm+fe-2 




n p»n 




L /=m+l J 



PjV,TO+fc-l,-,i ) nf(m), 
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where PN.m+k-i.-.j is j-th column of matrix Pjv,m+fe-i- Then we can get 



J5[(nf (m + k) - nf(m)) 2 \n N (m)} 

rm+k-2 



= ( n N (m) 



n p ^ 



l—m 



PN,m+k-i,;j ) +n N (m) 



m+k-2 

n p 

l=m 



N.I 



P: 



N,m+k—l,-,j 





rm+k-2 




(pN,m,i,- 


n ^ 


■PjV,m+k-l,-,j^ 




L /=m+l 




-m+k 


-2 i 





(3.3) 



2n iv (m) 



n p ^ 



PN.m+k-i,-,^, (m) + (n (m)) 



For eac/i AT > 1, we define the transition operators on { - ^ : k > 0} as follows 



S N , k f(p) = E[f{ K — '-)\—^ = pj 



iV 



(3.4) 



/or eac/i fc > and f 6 C(Kn), p € K 



N • 



Lemma 3.1. Define -Pjv,*> andX N in subsection \2.l\ andS^.k by \ 'J-4\I - If^O-N—toodvi^Nt-^) 
0, where djj is defined by then 



lim sup sup liV^jjvt] - /]/(p) - G A (t)/(p)| = 

AT-»oo <t<X peKTjv 



(3.5) 



/or any f eD = C 2 {K). 



Proof. Let f e D ~ C 2 {K), fix T > 0. For < i < T, and p e note that by 
Taylor's expansion and l|3.2(l 

[S'tv.iat*] ~ I}f(p) 

= Ei(x N (t + ±)-p)^-( P y\x N (t) = p} 



E[(X N (t + i) - p)^(Xf (i))(X w (t + ±) - p)'|X w (i) = p 



'ax 2 



i 



1 „ w 



E[(X N (t 



1> 
AT y 



p)^(x?(t))(x"(t + -) - P )'|x w (t) = P ], 



where Xf (t) = p + 8? {X N (t + i) - p), for some 6>f G (0, 1). Then 
|7V[S W] -/]/(p)-G.4(i)/(p)| 

<l P [^(M)-^)]^(p)'i 



N\E[(X"(t+-) -p)g(X^/i!!X,M/ f _i-p,'|X/N/i = P ] 



1 



1 



AT 



<9x 2 



AT ' 



(3.6) 
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Denote by I±(N,p,t) and l2(N,p,t), the first and second term on the right hand 
1| =max 1 < iJ < r Up- 



side of Let Hf^rll = maxi<ij< r ||^^-||- By Holder inequality 



/ 2 (iv,p ! t)<||g||iv^^£;[|(xf(t + l)-p l )(^f^ + ^)-p J )l|x JV W = p] 
»=i j=i 

^ iiSii^EE^k^ + ^) -pOW*) = p] 

1=1 j=l v 

1 

xE[(X»(t+±)- Pj ) 2 \X N (t)=p] 

Since Ajv(i) is a constant on [-^, for each k > 0, for fixed 1 < j < r, by (|3.3|) . wc 
get 

N 2 El(X?(t+±)- Pj f\X N (t)=p} 

=p^,., ( M) + (pa,,., ( M ))2 _ ^(M) _ £ f 

fe=i 



< E P < IAjv.IJ (t) | + (J2 Pl\ A N,l,3 (*)!)*+ 2 Pi \<*N,j,j (t) | 
i=l !=1 

r r 

< E i^Wj-wi + E i^mjWI) 2 + 2|o JVJJ (t)|. 



i=i i=i 

Since limjv^oo d[/(^Ar, -4) = and A is bounded on [0, T], there exists Ct > such 
that 

N 2 E[(Xf(t + 1) - Pj ) 2 |X iV (i) = p] < C T 

for any N > 1, < * < T, p € K N , and 1 < j < r. Then 

sup sup h(N,p,t) <l||^||r 2 C T . (3.7) 

Thus limjv^oo sup < t < T sup peKN I 2 {N, p, t) = 0. 

For matrix B = (bij) r xr an d vector y = (y%, ■ ■ ■ ,y r )', let ||£>|| = J2i j=i an d 
||y|| be the Euclidean norm of y. Then \\By\\ < \\B\\ ■ ||y||. It follows that 

h(N,p,t) < llp'H • \\A N A - A(t)\\ ■ &P)'\\ 

[Nt] 8f (3 ' 8) 

<ll^(^)-4(t)i|.||£( P )'ii. 

Notice that A N (^) = A N (t), t > 0, thus 

lim sup sup Ii(N,p,t) = (3.9) 

N ^°°0<t<Tp£K N 
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follows by (|3.8|) and liniAr^oo du{Ajsr, A) = 0. Therefore (|3.5[1 is proved. 
Corollary 3.1. With the same conditions as those in Lemma Vi.lV we have 



sup sup sup \N[S N t Nt i -I]f (p)\<oo (3.10) 

N 0<t<TpGK N 



and 



lim sup sup \[S Nt[Nt ] - I]f (p)\ = (3.11) 

A, - >oo 0<t<TpS-R"jv 



for anyT > and f £ C 2 {K). 

Proof. This follows from Lemma 13. II immediately. 

Remark 3.1. We can use Taylor's expansion of order 1 to prove directly that l|3.10fl 
and (P?TTT|) hold for any T > and / £ C l {K). 

Lemma 3.2. Define the time- dependent generator {GA(t),0 <t< oo} on C 1 (-ftT) by 
l\2.5\) . Let [i £ &{K). Then the Dk[0, oo) martingale problem for (Ga, A 4 ) has at most 
one solution. 

Proof. The limit process X°° of Theorem 12.11 is deterministic such that (X 00 )' 
satisfies the linear differential equation 

dx(t) 



dt 



A'(i)x(i). 



Then we have the uniqueness of the Dx[0, oo) martingale problem for (Ga, m)- 

Proof of Theorem 12.11 Part 1). This part follows by Lemma 13.11 13.21 and 
Remark IA.1I 

Next we make preparations for proving Theorem 12. II Part 2). 

At first, we consider the multiagent model with only multinomial sampling (MAMWMS). 
This is illustrated by Figure [5] 



N 


Multinomial 
Sampling 


N 







Figure 2: The transition structure of MAMWMS 
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This model is similar to the neutral Wright-Fisher model of population genetics. 
However in contrast to the standard genetics model here the internal state change 
(which would correspond to mutation) can be influenced by an external random envi- 
ronment. If we don't include in the Wright-Fisher model selection and gene mutation, 
we get a model which is very close to MAMWMS. We define a transition operator 
T/v related to the homogeneous Markov chain { " fc = 0, 1, - • • } on C{Kn) as 



follows: 



T N f(p) = E[f( \ T 2 > )\— ^ = p], / e C{K N ), p G K N . 



N N 

Let 

Z N (t) 



X;m _n N {[Nt] + \) 



N 

and let Z be a diffusion process in K with the generator Gb defined by (|2.7[) . Then 
we have the following analog of the classical Wright-Fisher diffusion limit. 

Proposition 3.1. With the conditions above, 

lim sup \N(T N -I)f(p)-G B f(p)\ = (3.12) 

N ~* °°pGK N 

for every f G C 2 {K). If Z N (0) =S> Z(0) m if, tfien Z w => Z m Z) K [0,oo). 
Define {t/jv fc, A: > 0} as follows: 

n^ffr -4- 1 "I n N (k) 

u N , k f( P ) = E[f( y = pL 7 e c ™' p e ^ for any fc - °- 

Then by the Figure ^ since and Tjv are one step transition operators related to 
the internal dynamics and the global multinomial sampling respectively, we have 

UN,k — Sn^Tn- (3.13) 

Lemma 3.3. Assume that lini/v— >oo du{Ajsr, A) = 0. With the definitions above, we 
have 

lim sup \N[U NAm] -I}f(p)-G AB (t)f(p)\ = (3.14) 
for every f G C 3 (K) and t>0. 
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Proof. Let / G C 3 {K), and fix t > 0. Choose T such that T > t, and p G 
notice that by (|3 . 1 3|) 

|iV[C/ W)[iVt] -7]/(p)-GAB(t)/(p)| 
= ([^[^(Tjv - 7)]/(p) - G B f(p) + JV[^,[jv*] " I]f(p) ~ G A (t)f(p)\ 
< \S NAm] [N(T N - I)}f(p) - S NAm] G B f(p)\ + \S N , [m] G B f(p) - G B f(p)\ 

+ \N[S N , m -I}f(p)-G A (t)f(p)\. 

It follows by (S/vjATt] being a contraction that 

sup \N[U Nj[m] - I]f(p) - G AB (t)f(p)\ 
< sup \[N(T N -I)]f(p)-G B f(p)\+ sup \S NAm] G B f(p)-G B f(p)\ 

peK N p£K N 

+ sup \N[S Nt[Nt] -I]f(p)-G A (t)f(p)\. 
peK N 

(jTTljl is then proved by (|3~TT1) . Remark O and (|3~5) . 

Lemma 3.4. FFif/i i/ie same conditions as those in Lemma \3.'A we have 

sup sup sup \N[U Nt[Nt i -J]/(p)| < oo (3.15) 

/or anyT>0 and f G C 3 (/sT). 

PVoo/. Let / G C^Jf ), fix T > 0. For < t < T, and p€K N , notice that 

sup |iV[C7^ )[m] - 7]/(p)| = sup \[NS Nt[m] (T N - I)]f(p) + N[S Nt[m] - I]f(p)\ 

p£K N p£K N 

< sup \N[T N -I]f(p)\+ sup \N[S NAm] -I]f(p)\. 

peK N p£K N 

jXTg} follows by (|3~TUjl and JSHJ. 

Now we introduce some notations before we prove the uniqueness of the martingale 
problem for (G AB ,fi), G &{K) on C 3 (K). For each n > 1, let N„ = {a : a G 
(Z + ) r , 5Zl=i a » = n l- ^ is clear that N n has c n = ( n+ 'j7 1 ) elements. Define an order 
' >' on N„ as follows: let a, a' G N„, a > a' <^ ot\ > a[ or there exists 2 < k < r, 
such that aj = a[ for 1 < I < k — 1 and > aj... We use N n as the index set when 
we define D^ n x Cn [0, oo)-valued matrix functions. For given A G D^yr [0, cx)), define 



*£&(*) = < 
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£ D R c n xc n [0, oo ) as follows: for each a, a' £ N„ and t > 0, 

ctjCikjit), if there exist 1 < j, fc < r, j ^ fc, such that a/ = a|, 

for 1 < I < r satisfying I ^ j or k, and atj — a 3 ■ — 1, 

°4 = a k + 1; 

0, otherwise. 

(3.16) 

We arrange the elements of along the rows and columns decreasingly by the order 
' >'. For each n > 1, we define a c„ x c„ matrix — (6^/) Q , a 'eN„ as follows: for 
each a, a' £ N n , 

e=( i(n_n2) ' ifa=a ' ; ( 3 - 17 ) 

I 0, otherwise. 
For each n > 2, using N n and N n -i as the index sets, we define a c n x c„_i matrix 
a€N„,a'£i„_i & s follows: for each a £ N n , a' G N n _i, 

,(n) I ^ a t( a i ~ 1)- if a- = a l - 1 and = a ; for / ^ i, 1 < / < r; 
I 0, otherwise. 

We also arrange the elements of B'"', £)( n ) along the rows and columns decreasingly 
by the order ' >'. 

Lemma 3.5. Define the time- dependent generator {GAB(t),0 <t< oo} on C 2 (K) by 
h2. &)) . Let \i £ ^(if). TTien i/ie -Dif[0, oo) martingale problem for (Gab, m) 071 C 3 (if) 
/ias at most one solution. 

Proof. Assume that {Y(t) = (Yi(t), ■ ■ ■ , Y r (t)) : < t < oo} is one solution of the 
martingale problem for (Gab,IJ>) on C 3 (K). Let n > 1, for arbitrary a £ N n , and 
t > 0, define f a (xi, • • ■ , x r ) — x" 1 ■ ■ ■ x" r and y"(t) — E[f a (Y(t))]. Define the column 
vector y n (t) = (j/"(t)) Q£ N„, where the elements of y n (t) is arranged decreasingly by 
the order ' >'. 

Then, for n > 1 and given a £ N„ and £ > 0, we have 

E[f a (Y(t))]=E[f a (Y(0))}+ f E[G A B(s)f a (Y(s))]ds. (3.19) 

Jo 

At first, for n = 1, since Gsfa =0 for any a £ Ni, we have 

E[f a (Y(t))}=E[f a (Y(0))}+ [ E[G A (s)f a (Y(s))}ds, 
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which implies 

Yi(t) =Yi(0)+ / A^(s) yi (s)ds. (3.20) 
Jo 

Next, we calculate £[/„(Y(t))] for n > 2. By (E7J), (|3~T7|) and ljXT%)) . we can get 

E[G B f a (Y(s))} 

i= 1 1 i— 1 j=lj^i 1 3 

i=l 

1 r r 

-2E E ^[«^y 1 ( s ) ai ---y r ( s ) a -] 
= i E - i)^[^i(*) Ql ■ • • W*- 1 • ■ ■ W*1 

i=l 

r r r 

-oE^-^ + E E aia 3 Wi(«r-y r (an 



(3.21) 



r r 



r r r 

= o^yn-i^) - -E E a ^ - E ^m^r • • • wi 



i=l J=l 



- ^(« 2 - n)E\Yi{ a ) ai • ••y r («) ar ] 

= J Di" ) yn-i(.s) + si , ; ) yn( S ), 

where is the a-row of the matrix DM, B^? the a-row of the matrix B^ n \ By 
(|3.16() we can get 



1 



E[G A (s)f a (Y(s))]ds = 



E[Y(s)A(s)^(Y(s)Y]d 



■s 

r k—1 



E a^Mvzis) + e E (^ (Q1 '"' " 1 - ak+1 - 

fc=l i=l 



■J=l 

r r 



ri.s 



+ E E « J ^(^ (Qi '-' Qfc+v -^- 1 '-' Q " ) w 
fe=i j=fe+i 

= f A^( S )y n (s)ds, 
Jo 

(3.22) 

where Ai™? is the a-row of the matrix A (ll) . Then by lpTT§)l . J^TJ, and (l3~2~2"l) . for 

n > 2, we obtain 



yn(i)=yn(Q)+ [ D^y n _ 1 (s)ds+ f [A^is) + B^]y n {s)d 



(3.23) 
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Define for n = 1, 



ifO<t<s<oo 



0, otherwise; 



and for n > 2, 



Vn(s,f) 



>!(")(*) if0<t<s<oo, 



0, 



otherwise. 



Let /i(<) = yi(0), < t < oo. Then, by (|3.20() and the theory of Volterra equations of 
the second kind (see e.g. Smithies (1958) or Tricomi (1957)), we know that xi = yi is 
the unique solution of the Volterra equation of the second kind 



Jo 

on the space L 2 ([0, T], R Cl ) for any T > 0. We can do this procedure recursively. 
Assume that we know that for n > 1 the unique solution x n = y„ is determined, 
then we define f n+1 (t) = y„+i(0) + f* D(" +1 )x n (s)ds. Then by we know that 

x„ +1 = y„+i is the unique solution of the Volterra equation of the second kind 



on the space L 2 ([0, T], R Cn+1 ) for any T > 0. Note that the construction of the Volterra 
equations does depend just on y n (0) for n > 1, and does not depend on y ra (i) for 
t > 0, n > 1. Then we conclude that all moments of the one-dimensional marginal 
distribution of any two solutions of the martingale problem for (Gab, f) are the same. 
The uniqueness of the martingale problem for (Gab,h) on C 3 (K) then follows by 
Theorem 4.2, Chapter 4 of Either and Kurtz (1986). 

Proof of Theorem EH Part 2). This part follows by Lemma ET3I EH1 1531 and 
Corollary [0 



4.1. Measurability with respect to random environment 

At first, for $ and VP define in Subsection 12. 21 we have the following Lemma. 
Lemma 4.1. <f> and are continuous mappings from (J2? c ,du) to (^(Dk[0, oo)), p). 




(3.24) 



x(s) = f n+1 (s) + / V n+l {s,t)x{t)dt, (0 < s < T) 




4. Multiagent models in random environment 
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Proof. This is immediate by (f?T5|) . (f2~%|) and Corollarv fA.il 

Now we consider the measurability related to A if A is an ££ c - valued process. 

Lemma 4.2. ^ssttme that A is an ££ c -value& process defined on some probability 
space (Cl,^,Q). Then (A,<f>{A)), (A,V(A)) are & /[3S(& C ) ® @(&(D K [Q,oo)))]- 
measurable. 

Proof. Since K is separable, so are Dk[0,oo) and £P(Dk[0, oo)). Since Jz? c is also 
separable, to prove that (A, $(A)) is measurable, it suffices to prove that if C g 33(Jzf c ), 
D g @(&>(D K [0,oo))), (A, $(A)) _1 (C x D) g ^. This is clear by Lemma fO and 
(A,$(A))- 1 (C x D) = j4 _1 (C n Similarly, we can prove that is 

measurable. 

Next, we consider the measurability related to {^4jv}. Since {A^(jr),k > 0} 
determines the transition structure of a Markov chain by {Sn,1c, fc > 0} in MAMWID 
or by {UN,k,k > 0} in MAMWID AMS, it is clear that $jv and ty N are continuous 
mappings from ££n to &{Dk n [0, oo)). Then we have the following lemma. 

Lemma 4.3. Assume that An is an ^f^-valued process defined on some probability 
space {Vt Nl ^ N ,Q N ). Then (A N , $ N {A N )), and (A N ,^ N (A N )) are & /\3$(J£ N ) ® 
38{^ > {D Kn [0, oo)))]-measurable. 

Remark 4.1. Using the notations $at, '5 at, <&, and 'J, we can restate Theorem l2.1l as 
follows. Let A N g if at, and A g J/f c . Let fi N g <?>(K N ), and \x g g?{K). Assume that 
limjv— ►(» d(Aisf, A) = 0, and hn M- 

1) MAMWID. Define <f> N {A N ) g £?(Dk n [0, oo)) and g ^{D K [Q, oo)), such 
that ^(l^Z^O) -1 = /ijv and ^(A^O)- 1 = /i. Then 

limjv^oo p(*jv(Ajv), $(!)) = 0. 

2) MAMWID AMS. Define ^n{An) g ^(-D^ [0, oo)) and *(i) g ^(L> K [0, oo)), 
such that * A r(i A r)Z Ar (0)~ 1 = fi N and ^(i)Z(O)- 1 = /i. Then 

lirn^oo p(V N (A N ), *(i)) = 0. 

4.2. Proof of Theorem fOl 

The next two lemmas will be used in the proof of Theorem 12.21 
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Lemma 4.4. Let A = ( ££ c -valued process a.s., defined on some probabil- 

ity space (ft, J- , P), and A = (ai.j)rxr be a C R rxr [0, oo)-valued process defined on some 
probability space (Cl,^,P). If PA" 1 — PA" 1 , then A is also an J£ c -valued process 



a.s. 



Proof. The proof of this lemma is straightforward. 
Lemma 4.5. Let An be an Jzf/v -valued process defined on some probability space 



Proof. We omit this proof. 

Lemma 14.41 and 14.51 indicate that the conditions specified at the beginning of sub- 
section f° r the processes {^4 at} and A just depend on the distributions of {^4 at} 
and A. 

Proof of Theorem 12.21 We just prove part 1). Part 2) can be proved similarly. 

At first, we prove (|2.1U|) . Since {Ajv} are D Rr xr [0, oo)-valued processes, A is a 
Cfjrxr[0, oo)-valued process, and An =>■ A, by Skorohod Representation theorem, there 
exist some probability space (O, & ', Q) and a sequence of D R r* r [0, oo)-valued processes 
{Aat} and a C^xr [0, oo)-valued process A satisfying limjv^oo g?(Aat, A) = Q-a.s, 
QA" 1 = QA' 1 , and QA^ 1 = QnA^ 1 on &(D Rrxr [0, oo)) for each N > 1. By Lemma 
14.41 and 14.51 A is an ^f c -valued process, and An is an Jzfjv -valued process for each 
N > 1. By Lemma IO and 14.31 (A,$(A)) and (A N , $ N (A N )) are measurable. By 
Remark 14. II we have 



For each / € C(Jz? c ), and g £ C(£P(D R [0, oo))), by bounded convergence theorem 



(Q,n , <^n ,Qn) a.s., and let A^ be a D Rr x r [0, oo)-valued process defined on (Qn, ^n,Qn)- 
If QnAJj 1 = QnAJj 1 , then An is also an J£n -valued process a.s. 



lim p{$ N (A N {u))),$(A(u>))) = 0, Q-a.s. 



(4.1) 




lim / f(A N (u) N ))g($ N (A N (u N )))Q(dw N ) 



lim / f(A N (u>))g($ N (A N (w)))Q(du) 
[ f(A(u))g($(A(u)))Q(dcj) 
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Since S£ c and ^(Dfc[0, oo)) are separable, G(Jzf c ) and G(^(Z?r-[0, oo))) are conver- 
gence determining on (Jz? c ,aV) and {^{Dk [0, oo)), p) respectively, (I2.1UI) is proved. 
Secondly, we prove that (|2.11() . By (|4.1|l . for any open set G C Dk[0, oo), we have 

liminf ® N (A N (u>))(G) > $(i(d)))(G), Q-a.s. (4.2) 

Then by Fatou Lemma, we get 

liminf / <P N (A N (d>))(G)Q(dcj) > [ $(!(£)) (G)Q(dtD), (4.3) 

N^oo J J 

which implies that 

liminf / <P N (A N (iu N ))(G)Q N (dLJ N ) > f $(A(u)))(G)Q(du>). (4.4) 

N-,oo J J 

Since Dk[0, oo) is separable, l|2.11|l is proved. 

Appendix A. Weak Convergence Criteria for Time Inhomogeneous 

Markov Processes 

In this section we state the weak convergence criteria for time inhomogeneous 
Markov processes. These criteria are concerned with martingale problems with time- 
dependent generators. We introduce some notations from Either and Kurtz (1986). For 
n = 1, 2, ■ • • , let {^"} be a complete filtration, and let ££ n be the space of real- valued 
{£f"}-progressive processes £ satisfying 

sup E[\£(t)\] < oo 

0<t<T 

for each T > 0. Let srf n be the collection of pairs (£, ip) € Jz? n x «5f„ such that 

Jo 

is a {^™}-martingale. 

Proposition A.l. Let (E,r) be a Polish space. Let {G(s),0 < s < oo} be a family of 
operators on C{E). Suppose that there exists a countable set T\ C [0, oo), such that for 
each s ^ r 1; {G(s),0 < s < oo} has a common domain denoted by 9(G) and for each 
f e 9(G), G(s)f £ C(J5) /or s ^ r x , and ||G(s)/|| is bounded for s e [0,T] \Ti for 
any T > 0. Suppose that there is an algebra C a contained in the closure of 9(G) (in 



Multiagent models in time-varying and random environment 



21 



the sup norm) which separates points. Suppose that the De[0, oo) martingale problem 
for (G,v) has at most one solution, where v £ ^(E). Suppose for each n > 1, X n 
is a {5^"}- adapted process with sample paths in De[0,oo). Suppose P(X n (0)) =>■ v 
and the compact containment condition holds. Suppose M C G{E) is separating. Then 
condition (a) implies that there exists a solution X of the De[0, oo) martingale problem 
for (G, v), and X n =>■ X : 

(a) There exists a countable set T2 C [0, 00) such that for each f £ S>{G), and 
T > 0, there exists i^m^n) £ s^m such that 

sup sup £[|£„(s)|] < 00, (A.l) 

n 0<s<T,s^Ti 

sup sup £?[|</J ra (s)|] < 00, (A. 2) 

n 0<s<T,s^Ti 

k 

lim E[(£ n (jt) - f{X n {t))) TT^(X n (ti))] = 0, (A.3) 

n — >oo A A 

i=l 

k 

lim E[(<p n (t) - G{t)f{X n {t))) TT hi(X n {U))} = 0, (A.4) 

n— *oo A A 

i=l 

for all k > 0, < h < t 2 < ■ ■ ■ < t k < t < T with U, t £ T x UT 2 , and hi, ■■■ ,hk & M, 
and 



lim £[ sup |£„(i) - /(X n (i))|] = 0, (A.5) 
™^°° tGQn[0,T] 

supSfH^nllp^] < 00, /or some j) G (l,oo], (A. 6) 



where Q is a countable and dense subset of R, \\h\\ Pt T — [Jq \h(t)\ p dt] 1 / p if p < 00; 
\\h\\oo,T = esssup < t < T \h(t)\. 

The conditions of the following two corollaries are more convenient to be verified for 
our multiagent models. 

Corollary A.l. Suppose in Proposition \A.1\ that for each n > 1, X n is a 
adapted process with sample paths in De[0, 00) and generator {G n (s), < s < 00} on 
G(E). Suppose also for each n > 1 there exists a countable set T n C [0, oo), such 
that for each s ^ T n , {G„(s),0 < s < 00} has a common domain denoted by Sl{G n ). 
Then condition (b) implies that there exists a solution X of the De[0, 00) martingale 
problem for (G, v), and X n =>• X : 
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(b) For each n > 1, Sl{G n ) = Sl{G) and there exists a countable set T2 C [0, 00) 
such that for each f G @(G), and T > 0, 

sup sup ||G n (s)/|| < 00, 

n 0<s<T,s£riUr™ 

where \\ ■ \\ is the sup norm on B(E), and 

lim sup\G n (t)f(q)-G(t)f(q)\ = 0, 

qeE 

for any t satisfying t <£ Ti U T 2 U™ =1 T n and < t < T . 

Corollary A. 2. Suppose in Proposition \A . 1\ that X n = f] n (Y n ([a n -])) and {Sf™} = 
{=^j^ n 4 i}j where {Y n (k),k = 0, 1,2, ••■} is a time inhomogeneous Markov chain in a 
metric space E n with transition functions pL n ^{x,Y), r/ n : E n 1— > E is Borel measurable, 
and a n — > 00 as n — > 00. Define T n ^ : B(E n ) 1— > B(E) by 

T n ,kf{x) = J f{y)li n ,k{x,dy), 

and let G n ^ = ct n (T n ^ — I). Then condition (c) implies that there exists a solution X 
of the De[0, 00) martingale problem for (G, v), and X n => X : 

(c) There exists a countable set T2 C [0, 00) such that for each f G 2>{G), and 
T > 0, 

sup sup sup \G ni [ ant ]f o r) n (q)\ < 00, (A.7) 
n o<t<T,t^riur 2 q&E n 

and 

lim sup \G ni [ ant ]f orj n (q) - G(t)f orj n (q)\ = 0, (A.8) 
for any t satisfying t ^ Ti U T2 and < t < T . 

Remark A.l. 1) Assume that D C ^(G), and C a C D where C a is a subalgebra 
of C(E) which separates points. If we can prove that the uniqueness of De[0, 00) 
martingale problem for (G, v) holds for functions in D instead of the common domain 
@(G), then we can replace @(G) by D in the conditions (a), (b) and (c). 2) We can 
simplify the condition (c) of Corollary IA.2I into the following version: there exists a 
countable set T2 C [0, 00) such that for each / G @(G), and T > 0, 

lim sup sup |G„,[ 0n t]/ or) n (q) - G(t)f or) n (q)\ = 0. (A.9) 

™-»°° 0<t<T,t^riUr 2 q€E„ 

This condition is stronger than the condition (c) , since l|A.9|) implies l|A.7|) and i|A.8|) . 
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